Welcome to Linear Models

Dr. Lucy D’Agostino McGowan

👋 Lucy D’Agostino McGowan

Find me: Manchester 342

“truth … is much too complicated to allow anything but approximations”

- John von Neumann



John von Neumann, wearer of funny hats via https://farkasdilemma.wordpress.com/2013/01/04/john-von-neumannm-wearer-of-funny-hats/

“All models are wrong, but some are useful”

- George Box





DavidMCEddy at en.wikipedia CC BY-SA 3.0 , via Wikimedia Commons

What is a linear model?

Is this a linear model?

What is the equation?

\[y = \beta_0 + \beta_1 x + \varepsilon\]

Is this a linear model?

Where is \(\beta_0\)

Is this a linear model?

Where is \(\beta_0\)

Is this a linear model?

Where is \(\beta_1\)

Is this a linear model?

Where is \(\beta_1\)

What is the equation?

\[y = \beta_0 + \beta_1 x + \varepsilon\]

What is the equation?

\[ X = \begin{bmatrix}1 & x_1\\ \vdots & \vdots \\ 1 & x_n\end{bmatrix},\quad \beta = \begin{bmatrix}\beta_0\\ \beta_1\end{bmatrix} \]

What does the “data” look like?


  (Intercept)        x
1           1 1.920595
2           1 3.093363
3           1 5.301387
4           1 8.990286
5           1 1.218501
6           1 8.882287

Is this a linear model?

What is the equation?

\[y = \beta_0 + \sum_{j=1}^{J} \beta_j B_j(x) + \varepsilon\]

  • where \(B_j(x)\) are known spline basis functions
  • the model is linear in the \(\beta_j\)

What does the “data” look like?


  (Intercept) ns(x, df = 3)1 ns(x, df = 3)2 ns(x, df = 3)3
1           1   -0.095842465      0.4743374     -0.3103069
2           1    0.006273703      0.5498909     -0.3597333
3           1    0.444366108      0.4236598     -0.2337245
4           1    0.133806217      0.3714757      0.4916902
5           1   -0.103895176      0.3825795     -0.2502798
6           1    0.164715184      0.3660870      0.4649746

Is this a linear model?

Is this a linear model?

What is the equation?

\[y = \sum_{k=1}^{K} \beta_k \, \mathbf{1}\{x \in R_k\} + \varepsilon\]

  • where \(R_1,\ldots,R_K\) are each of the regions
  • this is linear in the parameters \(\beta_k\)

What does the “data” look like?


  ind1 ind2 ind3 ind4 ind5 ind6 ind7 ind8
1    0    0    0    0    0    0    0    1
2    0    0    0    0    0    0    1    0
3    0    0    0    0    0    0    1    0
4    0    0    0    0    0    0    0    1
5    0    1    0    0    0    0    0    0
6    0    0    1    0    0    0    0    0

Is ChatGPT based on a linear model?

Are neural networks linear models?

Tech Setup